Skip to content

test(bench): add regression yardsticks and harden CI quality gate#101

Merged
hyp3rd merged 8 commits intomainfrom
feat/modernize
May 3, 2026
Merged

test(bench): add regression yardsticks and harden CI quality gate#101
hyp3rd merged 8 commits intomainfrom
feat/modernize

Conversation

@hyp3rd
Copy link
Copy Markdown
Owner

@hyp3rd hyp3rd commented May 2, 2026

test(bench): add regression yardsticks and harden CI quality gate

Add benchmark suites for ConcurrentMap, HistogramStatsCollector, and
HyperCache concurrency paths as regression yardsticks ahead of planned
performance optimizations (Phases 1–2). Capture an initial
bench-baseline.txt on Apple M4 Pro for benchstat comparison.

Harden CI and tooling:

  • Upgrade test-race to -count=10 -shuffle=on -timeout=15m in both
    Makefile and GitHub Actions workflow for stronger flake detection
  • Enable golangci-lint on test files (tests: true)
  • Add typecheck (go vet), build, and ci aggregate Makefile targets
  • Add bench-baseline capture target for ongoing benchstat comparisons
  • Remove stale golangci-lint badge link from README

hyp3rd added 3 commits May 3, 2026 00:18
Upgrade GitHub Actions tooling:
- Promote CodeQL workflow to Advanced setup with multi-language support
  (actions, go, ruby) and explicit build-mode per language
- Replace manual golangci-lint curl install with golangci/golangci-lint-action
  in the lint workflow
- Remove the redundant pre-commit CI workflow
- Bump golangci-lint to v2.12.1 and buf to v1.69.0 across project settings,
  Makefile, and pre-commit hook

Introduce internal/constants/errors.go to centralize HTTP JSON error labels
and messages (ErrorLabel, ErrMegMissingCacheKey,
ErrMsgUnsupportedDistributedBackend), replacing inline string literals
scattered across management_http.go, pkg/backend/dist_http_server.go,
and related tests.

Apply minor whitespace formatting improvements across multiple source files
to satisfy linter rules.
Add benchmark suites for ConcurrentMap, HistogramStatsCollector, and
HyperCache concurrency paths as regression yardsticks ahead of planned
performance optimizations (Phases 1–2). Capture an initial
bench-baseline.txt on Apple M4 Pro for benchstat comparison.

Harden CI and tooling:
- Upgrade test-race to -count=10 -shuffle=on -timeout=15m in both
  Makefile and GitHub Actions workflow for stronger flake detection
- Enable golangci-lint on test files (tests: true)
- Add typecheck (go vet), build, and ci aggregate Makefile targets
- Add bench-baseline capture target for ongoing benchstat comparisons
- Remove stale golangci-lint badge link from README
Copilot AI review requested due to automatic review settings May 2, 2026 23:41
@github-advanced-security
Copy link
Copy Markdown

You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool.

What Enabling Code Scanning Means:

  • The 'Security' tab will display more code scanning analysis results (e.g., for the default branch).
  • Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results.
  • You will be able to see the analysis results for the pull request's branch on this overview once the scans have completed and the checks have passed.

For more information about GitHub Code Scanning, check out the documentation.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to strengthen the repo’s performance-regression and CI/tooling story by adding benchmark yardsticks for upcoming optimization work, replacing hard-coded test ports with dynamic allocation in distributed/Merkle tests, and tightening lint/test workflow configuration.

Changes:

  • Added new benchmark suites for HyperCache concurrency paths, pkg/stats histogram collection, and pkg/cache/v2 concurrent-map behavior, plus a checked-in benchmark baseline.
  • Reworked several distributed/Merkle tests to use dynamically allocated loopback ports and cleaned up a number of test/benchmark lint issues.
  • Updated CI/tooling configuration: repeated shuffled race tests, new typecheck/build/ci Make targets, golangci-lint config changes, dependency/tool version bumps, and workflow updates.

Reviewed changes

Copilot reviewed 44 out of 46 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
tests/port_helper.go Adds a shared helper for allocating ephemeral loopback ports in tests.
tests/merkle_sync_test.go Switches Merkle sync test nodes from fixed ports to allocated ports.
tests/merkle_single_missing_key_test.go Replaces hard-coded distributed node ports with allocated ports.
tests/merkle_no_diff_test.go Replaces fixed node ports with allocated ports.
tests/merkle_empty_tree_test.go Replaces fixed node ports with allocated ports.
tests/merkle_delete_tombstone_test.go Replaces fixed node ports with allocated ports.
tests/integration/dist_rebalance_leave_test.go Simplifies a comment/assertion block around node-C ownership expectations.
tests/hypercache_mgmt_dist_test.go Updates management-endpoint tests to use shared error-label constants.
tests/hypercache_http_merkle_test.go Uses allocated ports for HTTP-backed Merkle sync test nodes.
tests/hypercache_distmemory_remove_readrepair_test.go Formatting-only whitespace adjustment.
tests/hypercache_distmemory_integration_test.go Formatting-only whitespace adjustment.
tests/hypercache_distmemory_hint_caps_test.go Formatting-only whitespace adjustment.
tests/benchmark/hypercache_set_benchmark_test.go Makes ignored Set errors explicit in benchmarks.
tests/benchmark/hypercache_list_benchmark_test.go Makes ignored Set errors explicit in benchmark setup.
tests/benchmark/hypercache_get_benchmark_test.go Makes ignored Set errors explicit in benchmark setup.
tests/benchmark/hypercache_concurrency_benchmark_test.go Adds new parallel HyperCache benchmarks for set/get/get-or-set/mixed workloads.
pool_test.go Formatting-only whitespace adjustment.
pkg/stats/histogramcollector_bench_test.go Adds new histogram collector benchmarks for hot write/read paths.
pkg/stats/histogramcollector.go Formatting-only whitespace adjustment.
pkg/eviction/clock_test.go Formatting-only whitespace adjustment.
pkg/eviction/cawolfu_test.go Formatting-only whitespace adjustment.
pkg/eviction/arc_test.go Formatting-only whitespace adjustment.
pkg/cache/v2/cmap_test.go Formatting-only whitespace adjustment.
pkg/cache/v2/cmap_bench_test.go Adds concurrent-map regression benchmarks for count/shard/iteration paths.
pkg/cache/v2/cmap.go Formatting-only whitespace adjustment.
pkg/cache/cmap.go Formatting-only whitespace adjustment.
pkg/backend/redis.go Formatting-only whitespace adjustment.
pkg/backend/inmemory.go Formatting-only whitespace adjustment.
pkg/backend/dist_memory.go Mostly formatting and lint-cleanup of ignored transport-call comments.
pkg/backend/dist_http_transport.go Formatting-only whitespace adjustment.
pkg/backend/dist_http_server.go Centralizes JSON error-label keys via shared constants.
management_http.go Centralizes management HTTP error-label/message constants.
internal/constants/errors.go Introduces shared string constants for common error payload keys/messages.
hypercache.go Formatting-only whitespace adjustment.
go.sum Refreshes dependency checksums after module/version updates.
go.mod Bumps several direct and indirect tool/runtime dependencies.
bench-baseline.txt Adds a checked-in benchmark output snapshot for comparison.
README.md Removes stale golangci-lint badge link reference.
Makefile Expands quality-gate targets and hardens race/benchmark commands.
.project-settings.env Updates shared tool version defaults.
.pre-commit/golangci-lint-hook Updates default golangci-lint hook version.
.golangci.yaml Adjusts enabled/disabled linters and linter settings.
.github/workflows/test.yml Hardens CI test step with repeated shuffled race runs.
.github/workflows/pre-commit.yml Removes the dedicated pre-commit workflow.
.github/workflows/lint.yml Reworks lint workflow setup and switches golangci execution method.
.github/workflows/codeql.yml Replaces basic CodeQL workflow with the advanced multi-language template.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/lint.yml Outdated
Comment thread .golangci.yaml Outdated
Comment thread Makefile
Comment thread Makefile
Comment on lines 34 to 36
# bench runs the benchmark tests in the benchmark subpackage of the tests package.
bench:
cd tests/benchmark && go test -bench=. -benchmem -benchtime=4s . -timeout 30m
Comment thread Makefile
Comment on lines +38 to +40
# bench-baseline captures the current benchmark output to bench-baseline.txt for benchstat comparison.
bench-baseline:
cd tests/benchmark && go test -bench=. -benchmem -benchtime=4s -count=5 . -timeout 30m | tee ../../bench-baseline.txt
Comment thread internal/constants/errors.go Outdated
Comment on lines +64 to +65
# with:
# version: "${{ steps.settings.outputs.golangci_lint_version }}"
hyp3rd and others added 5 commits May 3, 2026 01:49
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
… packages

Remove the global sync.RWMutex from HyperCache and the InMemory backend,
delegating thread-safety to the eviction algorithm's internal lock and
the sharded ConcurrentMap respectively.

Rewrite HistogramStatsCollector with per-stat atomic aggregates (count,
sum, min, max) and a bounded ring buffer (DefaultSampleCapacity=4096) to
fix the sort-under-RLock data race where GetStats sorted the live backing
slice in-place while concurrent writers appended to it.

Store the DistMemory transport in an atomic.Pointer<distTransportSlot> via
loadTransport/storeTransport helpers to prevent concurrent read/write races
across forwarding, replication, and hinted-handoff paths.

Additional fixes:
- NewConfig now returns (*Config[T], error) instead of panicking on an
  empty backendType; all call-sites updated including examples and benchmarks
- DistMemory.Stop is now idempotent and closes tombStopCh and
  rebalanceStopCh goroutines that were previously leaked
- WorkerPool.Enqueue silently drops jobs after Shutdown instead of sending
  on a closed channel, preventing panics during graceful cache teardown
- Suppress Fiber startup banner in management and dist HTTP servers to
  reduce -count=N test noise
- Fix typo ErrMegMissingCacheKey → ErrMsgMissingCacheKey
- Harden distributed tests with StopOnCleanup, polling loops instead of
  fixed sleeps, and increased heartbeat/merkle timeouts for -race runs

BREAKING CHANGE: NewConfig signature changed from *Config[T] to (*Config[T], error).
…ontention

Add `evictionShardCount` field to `HyperCache` and a corresponding
`WithEvictionShardCount` option (default: 32, matching pkg/cache/v2.ShardCount).

When `evictionShardCount > 1`, `initEvictionAlgorithm` wraps the selected
algorithm in `eviction.Sharded`, aligning each key's data shard with its
eviction shard to eliminate the single global eviction-algorithm mutex.
This trades strict global LRU/LFU ordering for significantly improved
parallel-write throughput (~4x on Apple M4 Pro: SetParallel drops from
~750 ns/op to ~200 ns/op). Values ≤ 1 preserve the previous single-instance
behaviour with strict ordering.

Additional changes:
- Replace `errors.New` with `ewrap.New` for ErrInvalidAddress in cluster/node.go
- Add benchmark result snapshots: bench-baseline-v2, bench-step1, bench-step1-unit, bench-step2
- Add lint baseline snapshot: lint-baseline-v2
… parallelize, remove pkg/cache v1

Migrate all test assertions from github.com/longbridgeapp/assert to
github.com/stretchr/testify/require (promoted from indirect to direct
dependency in go.mod). The longbridgeapp/assert package is fully removed.

Add t.Parallel() to every applicable test function across the codebase,
including hypercache_test.go, pool_test.go, pkg/eviction, pkg/stats,
pkg/cache/v2, and the full tests/ and tests/integration/ trees. Tests
that share a cache instance and depend on insertion order are explicitly
annotated with //nolint:paralleltest.

Remove pkg/cache v1 (cmap.go, item.go and their tests) entirely. The
CAWOLFU eviction algorithm now uses a plain map[string]*CAWOLFUNode
guarded by its own mutex instead of the redundant ConcurrentMap sharding.

Additional modernization:
- Fix golangci.yaml: add path-based exclusions for test-specific linters
  (funlen, cyclop, gocognit, dupl, gocritic, goconst, noinlineerr, revive
  function-length/cognitive-complexity/cyclomatic).
- Rename benchmark package declarations from `tests` to `benchmark`.
- Replace http.Get/client.Get with context-aware http.NewRequestWithContext.
- Use t.Cleanup for deferred Stop/Close calls to surface return errors.
- Wrap sentinel errors with %w; introduce errUnexpectedStatus.
- Rename local `cap` constants to `windowCap` to avoid shadowing the builtin.
- Use net.ListenConfig.Listen instead of net.Listen in allocatePort.
- Add nolint directives for gosec, revive, and prealloc where appropriate.
Replace two `//nolint:gosec` suppressed `uint32(...)` casts in `Sharded`
with explicit calls to `converters.ToUint32` from `github.com/hyp3rd/sectools`.

- `NewSharded`: compute shard mask via `converters.ToUint32(shardCount - 1)`
  and propagate any conversion error instead of silencing the lint warning.
- `Evict`: convert `len(s.shards)` via `converters.ToUint32`, returning a
  zero-value on error rather than casting unchecked.
- Add the `sectools/pkg/converters` import to `sharded.go`.
- Register unsharded in `cspell.config.yaml` to keep spell-check clean.
@hyp3rd hyp3rd merged commit b223bbf into main May 3, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants